Case study research for better evaluations of complex interventions: rationale and challenges.

BACKGROUND: The need for better methods for evaluation in health research has been widely recognised. The 'complexity turn' has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research-currently denigrated as poor evidence-is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear. MAIN BODY: Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports. CONCLUSION: Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

Seeing the Forest for the Trees: A Set of Descriptive Case Studies Presented with the Networks of Care Framework.

Durable solutions for daunting problems in global health can be elusive. The global health literature tends to present aggregated data and highlight clinical outcomes but fails to describe the systems that buttress the interventions. The common idiom about "missing the forest for the trees" is apropos: by focusing on individual examples, we may miss the bigger picture. How implementation of policies and innovations plays out on the front lines of service delivery often goes uncommunicated. The Networks of Care scoping study takes a different approach, looking at diverse programs to seek out common patterns. Using the four domains of the Networks of Care framework to structure descriptions of six operational programs reveals commonalities in their designs and shows the utility of the framework's components. The commonalities increase our conviction that the framework can be used as a practical approach to strengthen service-level health systems. The case studies are followed by a commentary about the potential synergy of Networks of Care with Universal Health Coverage efforts, to deliver on the core promises to increase access and quality of care for all, especially the persistently underserved. These case studies help define a practical toolkit to promote enduring positive changes, forging a path for the Networks of Care framework to move anecdotes of individual successes to health policy and broader implementation, enabling global health practitioners at all levels to keep the big picture in focus while working toward ensuring healthy lives and well-being for all.

The University of Cincinnati College of Nursing Case Study: Bias and Discrimination in the Nursing Learning Environment.

The purpose of this case study is to outline strategies employed by the University of Cincinnati's College of Nursing (CoN) to increase underrepresented racial and ethnic (URE), and economically and educationally disadvantaged student acceptance, presence, inclusion, and success. The case study method was used to examine strategies used at the CoN to address bias and discrimination, identify student success strategies for URE students, and outline college initiatives to facilitate an inclusive environment. CoN leadership has instituted several programs involving faculty and students in efforts to decrease bias and discrimination and promote inclusion. They continue to engage faculty and others in adding to and improving their efforts. This is a process of culture change and must involve everyone. CoN leadership is committed to both demonstrating by example and holding all accountable for progressively improved continued efforts to create a more inclusive environment.

Impact of overdose prevention sites during a public health emergency in Victoria, Canada.

The primary objective of this study was to examine the impacts associated with implementation of overdose preventions sites (OPSs) in Victoria, Canada during a declared provincial public health overdose emergency. A rapid case study design was employed with three OPSs constituting the cases. Data were collected through semi-structured interviews with 15 staff, including experiential staff, and 12 service users. Theoretically, we were informed by the Consolidated Framework for Implementation Research. This framework, combined with a case study design, is well suited for generating insight into the impacts of an intervention. Zero deaths were identified as a key impact and indicator of success. The implementation of OPSs increased opportunities for early intervention in the event of an overdose, reducing trauma for staff and service users, and facilitated organizational transitions from provision of safer supplies to safer spaces. Providing a safer space meant drug use no longer needed to be concealed, with the effect of mitigating drug related stigma and facilitating a shift from shame and blame to increasing trust and development of relationships with increased opportunities to provide connections to other services. These impacts were achieved with few new resources highlighting the commitment of agencies and harm reduction workers, particularly those with lived experience, in achieving beneficial impacts. Although mitigating harms of overdose, OPSs do not address the root problem of an unsafe drug supply. OPSs are important life-saving interventions, but more is needed to address the current contamination of the illicit drug supply including provision of a safer supply.

Perception of health-related case studies in the context of introduction to clinical medicine course: students' and teachers' perspective.

BACKGROUND: Integrating theoretical and practical knowledge and stimulating students' active learning is the most important task of modern and high-quality healthcare education. By analyzing clinical cases, undergraduate medical students are trained to make accurate diagnoses, to choose appropriate therapy based on laboratory results and on adequate diagnostic tests. AIM: To examine the effect of clinical cases presentations on short-term memory as well as on the student's and teachers' evaluation of this method of teaching and learning in undergraduate medical studies. METHODS: A total of 107 students in term 4 and term 5 enrolled in Trinity Medical Sciences University in St. Vincent and the Grenadines participated in this cross-sectional study. At the end of the semester, the students and the faculty were asked to complete a survey to assess their perception of case presentations (10 items, 4-point Likert-type scale, strongly agree to strongly disagree). The results of pre- and post-presentation quizzes were evaluated using the Wilcoxon signed rank test for paired samples. RESULTS: Term 4 and term 5 students significantly improved their achievement after intervention (Wilcoxon test Z = - 11.282, p < 0.001, and Wilcoxon test Z = - 10.167, p < 0.001, respectively). The analysis of progress among low- and high-performance students in both terms showed a significant increase in performance. Overall, median students' and teachers' attitude scores were positive. CONCLUSION: Clinical case presentation has a learning potential and facilitates positive interaction between instructors and students and supporting students to become reflective and competent physicians.

Reducing the length of clinic visits: A process analysis case report.

Because the University of Houston Student Health Center often had more demand for services than available appointments, the aim of this study was to streamline their clinic visit process. While appointments were scheduled for 20 minutes, the clinic's patient visit cycle time (from check-in to check-out) averaged 70 minutes. This report demonstrats how to perform process analysis using failure modes and effects analysis in order to identify the highest priority cause of waste and how to conduct a detailed evaluation of ideas using a prioritization matrix in order to select the best solution to help streamline a process. Analysis of this clinic's visit process identified that patients (mostly international students) asking many questions during their appointment regarding issues not directly related to their care led to long clinic visits. To address this issue, the clinic recruited a team of international students to create a frequently asked questions video with answers, and the final video was then uploaded to the clinic's Facebook page. Comparing before and after the video was posted showed the clinic's average time for a clinic visit was reduced by more than 15%, and it was able to accommodate nearly 100 additional patient visits per month.

Nursing resource team capacity planning using forecasting and optimization methods: A case study.

AIM: To estimate the cost-minimizing size and skill mix of a nursing resource team (NRT). BACKGROUND: Nurse absences can be filled by an NRT at lower hourly cost than staffing agencies or nurses working overtime, but an NRT must be appropriately sized to minimize total cost. METHODS: Using all registered nurse (RN) absences at an academic teaching hospital from 1 October 2014 to 31 March 2018, we developed a generalized additive model (GAM) to forecast the weekly frequency of each of ten types of absence over 52 weeks. We used the forecasts in an optimization model to determine the cost-minimizing NRT composition. RESULTS: The median weekly frequencies for the ten absence types ranged between 12 and 65.5. The root mean squared errors of the GAMs ranged between 4.55 and 9.07 on test data. The NRT dimensioned by the optimization model yields an estimated annual cost reduction of $277,683 (Canadian dollars) (7%). CONCLUSIONS: The frequency of RN absences in a hospital can be forecasted with high accuracy, and the use of forecasting and optimization to dimension an NRT can substantially reduce the cost of filling RN absences. IMPLICATIONS FOR NURSING MANAGEMENT: This methodology can be adapted by any hospital to optimize nurse staffing.

Mixed methods evaluation of a hospital group model using an embedded research approach: study protocol.

INTRODUCTION: Hospital group models represent an organisational form that aims to bring together multiple provider organisations with a central headquarters and unified leadership responsible for locally managed operating units, standardised systems and a value-set shared across the group. These models seek to improve outcomes by reducing unwarranted variations in care provision and reducing costs through economies of scale. There is limited evidence on the impact and processes of implementing these models, so this study aims to evaluate one case study of a hospital group model. METHODS AND ANALYSIS: We will conduct a formative, mixed-methods evaluation using an embedded research approach to analyse the implementation of the model and its impact on outcomes and costs. We will carry out a multisited ethnography to analyse the programme theory for model design and implementation, the barriers and facilitators in the implementation; and wider contextual issues that influence implementation using semi-structured interviews (n=80), non-participant observations (n=80 hours), 'shadowing' (n=20 hours) and documentary analysis. We will also carry out an economic evaluation composed of a cost-consequence analysis and a return on investment analysis to evaluate the costs of creating and running the model and balance these against the potential cost-savings. ETHICS AND DISSEMINATION: The study protocol was reviewed by the local R&D Office and University College London Ethics Committee and classified as a service evaluation, not requiring approval by a research ethics committee. We will follow guidelines for informed consent, confidentiality and information governance, and address issues of critical distance prevalent in embedded research. Findings will be shared at regular time points to inform the implementation of the model. The evaluation will also generate: an evaluation framework to evaluate future changes; recommendations for meaningful baseline data and measuring improvement; identification of implementation costs and potential cost-savings; and lessons for the National Health Service on implementing these models.

Incorporating Theory into Practice: Reconceptualizing Exemplary Care Coordination Initiatives from the US Veterans Health Delivery System.

This perspective paper seeks to lay out an efficient approach for health care providers, researchers, and other stakeholders involved in interventions aimed at improving care coordination to partner in locating and using applicable care coordination theory. The objective is to learn from relevant theory-based literature about fit between intervention options and coordination needs, thereby bringing insights from theory to enhance intervention design, implementation, and troubleshooting. To take this idea from an abstract notion to tangible application, our workgroup on models and measures from the Veterans Health Administration (VA) State of the Art (SOTA) conference on care coordination first summarizes our distillation of care coordination theoretical frameworks (models) into three common conceptual domains-context of an intervention, locus in which an intervention is applied, and specific design features of the intervention. Then we apply these three conceptual domains to four cases of care coordination interventions ("use cases") chosen to represent various scopes and stages of interventions to improve care coordination for veterans. Taken together, these examples make theory more accessible and practical by demonstrating how it can be applied to specific cases. Drawing from theory offers one method to anticipate which intervention options match a particular coordination situation.

Surveillance, anticipation and firefighting: Perspectives of patient safety from a New Zealand case study.

AIM: This study explored how doctors, nurses and managers working in a New Zealand tertiary hospital understand patient safety. BACKGROUND: Despite health care systems implementing proven safety strategies from high reliability organisations, such as aviation and nuclear power, these have not been uniformly adopted by health care professionals with concerns raised about clinician engagement. DESIGN: Instrumental, embedded case study design using qualitative methods. METHODS: The study used purposeful sampling, and data was collected using focus groups and semi-structured interviews with doctors (n = 31); registered nurses (n = 19); and senior organisational managers (n = 3) in a New Zealand tertiary hospital. RESULTS: Safety was described as a core organisational value. Clinicians appreciated proactive safety approaches characterized by anticipation and vigilance, where they expertly recognized and adapted to safety risks. Managers trusted evidence-based safety rules and approaches that recorded, categorized and measured safety. CONCLUSION AND IMPLICATIONS FOR NURSING MANAGEMENT: It is important that nurse managers hold a more refined understanding about safety. Organisations are more likely to support safe patient care if cultural complexity is accounted for. Recognizing how different occupational groups perceive and respond to safety, rather than attempting to reinforce a uniform set of safety actions and responsibilities, is likely to bring together a shared understanding of safety, build trust and nurture safety culture.

Research Methodologies for Total Worker Health®: Proceedings From a Workshop.

OBJECTIVE: There is growing interest in the NIOSH Total Worker Health program, specifically in the process of designing and implementing safer, health-promoting work and workplaces. A Total Worker Health (TWH) Research Methodology Workshop was convened to discuss research methods and future needs. METHODS: Twenty-six experts in occupational safety and health and related fields reviewed and discussed current methodological and measurement issues and those showing promise. RESULTS: TWH intervention studies face the same challenges as other workplace intervention studies and some unique ones. Examples are provided of different approaches and their applications to TWH intervention studies, and desired developments in the TWH literature. CONCLUSIONS: This report discusses and outlines principles important to building the TWH intervention research base. Rigorous, valid methodologic, and measurement approaches are needed for TWH intervention as well as for basic/etiologic, translational, and surveillance research.

Application of Constrained Optimization Methods in Health Services Research: Report 2 of the ISPOR Optimization Methods Emerging Good Practices Task Force.

BACKGROUND: Constrained optimization methods are already widely used in health care to solve problems that represent traditional applications of operations research methods, such as choosing the optimal location for new facilities or making the most efficient use of operating room capacity. OBJECTIVES: In this paper we illustrate the potential utility of these methods for finding optimal solutions to problems in health care delivery and policy. To do so, we selected three award-winning papers in health care delivery or policy development, reflecting a range of optimization algorithms. Two of the three papers are reviewed using the ISPOR Constrained Optimization Good Practice Checklist, adapted from the framework presented in the initial Optimization Task Force Report. The first case study illustrates application of linear programming to determine the optimal mix of screening and vaccination strategies for the prevention of cervical cancer. The second case illustrates application of the Markov Decision Process to find the optimal strategy for treating type 2 diabetes patients for hypercholesterolemia using statins. The third paper (described in Appendix 1) is used as an educational tool. The goal is to describe the characteristics of a radiation therapy optimization problem and then invite the reader to formulate the mathematical model for solving it. This example is particularly interesting because it lends itself to a range of possible models, including linear, nonlinear, and mixed-integer programming formulations. From the case studies presented, we hope the reader will develop an appreciation for the wide range of problem types that can be addressed with constrained optimization methods, as well as the variety of methods available. CONCLUSIONS: Constrained optimization methods are informative in providing insights to decision makers about optimal target solutions and the magnitude of the loss of benefit or increased costs associated with the ultimate clinical decision or policy choice. Failing to identify a mathematically superior or optimal solution represents a missed opportunity to improve economic efficiency in the delivery of care and clinical outcomes for patients. The ISPOR Optimization Methods Emerging Good Practices Task Force's first report provided an introduction to constrained optimization methods to solve important clinical and health policy problems. This report also outlined the relationship of constrained optimization methods relative to traditional health economic modeling, graphically illustrated a simple formulation, and identified some of the major variants of constrained optimization models, such as linear programming, dynamic programming, integer programming, and stochastic programming. The second report illustrates the application of constrained optimization methods in health care decision making using three case studies. The studies focus on determining optimal screening and vaccination strategies for cervical cancer, optimal statin start times for diabetes, and an educational case to invite the reader to formulate radiation therapy optimization problems. These illustrate a wide range of problem types that can be addressed with constrained optimization methods.

Physicians' Voices: What Skills and Supports Are Needed for Effective Practice in an Integrated Delivery System? A Case Study of Kaiser Permanente.

Payers are demanding that US health care become more accountable and integrated, posing new demands for physicians and the organizations that partner with them. We conducted focus groups with 30 physicians in a large integrated delivery system who had previous experience practicing in less integrated settings and asked about skills they need to succeed in this environment. Physicians identified 3 primary skills: orienting to teams and systems, engaging patients as individuals and as a panel, and integrating cost awareness into practice. Physicians also expressed a high level of trust that the system was designed to help them provide better care. This belief appeared to make the new demands and mental shifts tolerable, even welcome, standing in contrast to research showing widespread physician distrust of their institutional settings. Physicians' new skills and the system features that promote trust are described in the article and should be a focus for systems transitioning to a more integrated, accountable model.

A 'movement for improvement'? A qualitative study of the adoption of social movement strategies in the implementation of a quality improvement campaign.

Given the difficulties of implementing 'top-down' quality improvements, health service leaders have turned to methods that empower clinicians to co-produce 'bottom-up' improvements. This has involved the adoption of strategies and activities associated with social movements, with clinicians encouraged to participate in collective action towards the shared goal of improvement. This paper examines the adoption of social movement methods by hospital managers as a strategy for implementing a quality improvement 'campaign'. Our case study suggests that, despite the claim of empowering clinicians to develop 'bottom-up' improvements, the use of social movement methods can be more narrowly concerned with engaging clinicians in pre-determined programmes of 'top-down' change. It finds a prominent role for 'hybrid' clinical leaders and other staff representatives in the mobilisation of the campaign, especially for enrolling clinicians in change activities. The work of these 'hybrids' suggests some degree of creative mediation between clinical and managerial interests, but more often alignment with the aspirations of management. The study raises questions about the translation of social movement's theories as a strategy for managing change and re-inventing professionalism.

Hospital branding in Italy: A pilot study based on the case method.

The article investigates if, and in affirmative case how, Italian hospitals are managing corporate brand communication. Thanks to results of qualitative research, this article offers insights on Italian hospital branding. The pilot study based in the case method is to be considered a starting point for wider investigations on this topic, and it is useful for managers and practitioners who want to understand the role of corporate brand in hospital communication management and to connect health care professionals with the audience in a meaningful way in those countries in which the health care system is a mix of both public and private institutions.

Information Sharing: Best Practices That Support Transitions in Care.

Care coordination that improves patient care and patient outcomes is becoming increasingly necessary as bundled payment programs are developed. Rather than looking at each aspect of the patient's care, the entire care continuum from preoperative preparation through completion of the episode will become the norm. The length of the episode of care may be 30 days or as long as 90 days. The transition to different care providers during that episode requires information sharing. This is best accomplished by a technology platform that allows for real-time information sharing. This article describes one organizations experience with managing the full continuum of care.

Pathways to Program Success: A Qualitative Comparative Analysis (QCA) of Communities Putting Prevention to Work Case Study Programs.

OBJECTIVE: To examine the elements of capacity, a measure of organizational resources supporting program implementation that result in successful completion of public health program objectives in a public health initiative serving 50 communities. DESIGN: We used crisp set Qualitative Comparative Analysis (QCA) to analyze case study and quantitative data collected during the evaluation of the Communities Putting Prevention to Work (CPPW) program. SETTING: CPPW awardee program staff and partners implemented evidence-based public health improvements in counties, cities, and organizations (eg, worksites, schools). PARTICIPANTS: Data came from case studies of 22 CPPW awardee programs that implemented evidence-based, community- and organizational-level public health improvements. INTERVENTION: Program staff implemented a range of evidence-based public health improvements related to tobacco control and obesity prevention. MAIN OUTCOME MEASURE: The outcome measure was completion of approximately 60% of work plan objectives. RESULTS: Analysis of the capacity conditions revealed 2 combinations for completing most work plan objectives: (1) having experience implementing public health improvements in combination with having a history of collaboration with partners; and (2) not having experience implementing public health improvements in combination with having leadership support. CONCLUSION: Awardees have varying levels of capacity. The combinations identified in this analysis provide important insights into how awardees with different combinations of elements of capacity achieved most of their work plan objectives. Even when awardees lack some elements of capacity, they can build it through strategies such as hiring staff and engaging new partners with expertise. In some instances, lacking 1 or more elements of capacity did not prevent an awardee from successfully completing objectives. These findings can help funders and practitioners recognize and assemble different aspects of capacity to achieve more successful programs; awardees can draw on extant organizational strengths to compensate when other aspects of capacity are absent.

Setting healthcare priorities: a description and evaluation of the budgeting and planning process in county hospitals in Kenya.

This paper describes and evaluates the budgeting and planning processes in public hospitals in Kenya. We used a qualitative case study approach to examine these processes in two hospitals in Kenya. We collected data by in-depth interviews of national level policy makers, hospital managers, and frontline practitioners in the case study hospitals (n = 72), a review of documents, and non-participant observations within the hospitals over a 7 month period. We applied an evaluative framework that considers both consequentialist and proceduralist conditions as important to the quality of priority-setting processes. The budgeting and planning process in the case study hospitals was characterized by lack of alignment, inadequate role clarity and the use of informal priority-setting criteria. With regard to consequentialist conditions, the hospitals incorporated economic criteria by considering the affordability of alternatives, but rarely considered the equity of allocative decisions. In the first hospital, stakeholders were aware of - and somewhat satisfied with - the budgeting and planning process, while in the second hospital they were not. Decision making in both hospitals did not result in reallocation of resources. With regard to proceduralist conditions, the budgeting and planning process in the first hospital was more inclusive and transparent, with the stakeholders more empowered compared to the second hospital. In both hospitals, decisions were not based on evidence, implementation of decisions was poor and the community was not included. There were no mechanisms for appeals or to ensure that the proceduralist conditions were met in both hospitals. Public hospitals in Kenya could improve their budgeting and planning processes by harmonizing these processes, improving role clarity, using explicit priority-setting criteria, and by incorporating both consequentialist (efficiency, equity, stakeholder satisfaction and understanding, shifted priorities, implementation of decisions), and proceduralist (stakeholder engagement and empowerment, transparency, use of evidence, revisions, enforcement, and incorporating community values) conditions.

The Impact of an Online Crowdsourcing Diagnostic Tool on Health Care Utilization: A Case Study Using a Novel Approach to Retrospective Claims Analysis.

BACKGROUND: Patients with difficult medical cases often remain undiagnosed despite visiting multiple physicians. A new online platform, CrowdMed, uses crowdsourcing to quickly and efficiently reach an accurate diagnosis for these patients. OBJECTIVE: This study sought to evaluate whether CrowdMed decreased health care utilization for patients who have used the service. METHODS: Novel, electronic methods of patient recruitment and data collection were utilized. Patients who completed cases on CrowdMed's platform between July 2014 and April 2015 were recruited for the study via email and screened via an online survey. After providing eConsent, participants provided identifying information used to access their medical claims data, which was retrieved through a third-party web application program interface (API). Utilization metrics including frequency of provider visits and medical charges were compared pre- and post-case resolution to assess the impact of resolving a case on CrowdMed. RESULTS: Of 45 CrowdMed users who completed the study survey, comprehensive claims data was available via API for 13 participants, who made up the final enrolled sample. There were a total of 221 health care provider visits collected for the study participants, with service dates ranging from September 2013 to July 2015. Frequency of provider visits was significantly lower after resolution of a case on CrowdMed (mean of 1.07 visits per month pre-resolution vs. 0.65 visits per month post-resolution, P=.01). Medical charges were also significantly lower after case resolution (mean of US $719.70 per month pre-resolution vs. US $516.79 per month post-resolution, P=.03). There was no significant relationship between study results and disease onset date, and there was no evidence of regression to the mean influencing results. CONCLUSIONS: This study employed technology-enabled methods to demonstrate that patients who used CrowdMed had lower health care utilization after case resolution. However, since the final sample size was limited, results should be interpreted as a case study. Despite this limitation, the statistically significant results suggest that online crowdsourcing shows promise as an efficient method of solving difficult medical cases.